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Abstract. We apply to the semantics of Arithmetic the idea of "finite approximation" 
used to provide computational interpretations of Herbrand's Theorem, and we interpret 
classical proofs as constructive proofs (with constructive rules for V, 3) over a suitable 
structure A/" for the language of natural numbers and maps of Godel's system T. We 
introduce a new Realizability semantics we call "Interactive learning-based Realizability" , 
for Heyting Arithmetic plus EMi (Excluded middle axiom restricted to E? formulas). In- 
dividuals of M evolve with time, and realizers may "interact" with them, by influencing 
their evolution. We build our semantics over Avigad's fixed point result, but the same 
semantics may be defined over different constructive interpretations of classical arithmetic 
(Berardi and de' Liguoro use continuations). Our notion of realizability extends intuition- 
istic realizability and differs from it only in the atomic case: we interpret atomic realizers 
as "learning agents". 



1. Introduction 

From now on, we will call HA Heyting Intuitionistic Arithmetic, with a language in- 
cluding one symbol for each primitive recursive predicate or function. We call X^-formulas 
the set of all formulas 3x.P(x, y) for some primitive recursive predicate P, and EMi the 
Excluded middle axiom restricted to T,® -formulas. For a detailed study of the intuitionistic 
consequences of the sub-classical axiom EMi we refer to pQ . 

In this paper we give the full version of Aschieri and Berardi [2] and we extend Berardi 
and de' Liguoro ([5], [8]) notion of atomic realizability - originally conceived for quantifier 
free primitive recursive Arithmetic plus EMi - to full predicate logic, namely Heyting Arith- 
metic with EMi (HA + EMi). Our idea is to interpret classical proofs as constructive proofs on 
a suitable structure M for natural numbers and maps of Godel's system T, by applying to 
the semantics of Arithmetic the idea of "finite approximation" used to interpret Herbrand's 
Theorem. We extend intuitionistic realizability to a new notion of realizability, which we 
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call "Interactive learning-based Realizability" . We provide a term assignment for the stan- 
dard natural deduction system of HA + EMi, which is surprisingly equal in all respects to 
that of HA, but for the fact that we have non-trivial realizers for atomic formulas and a new 
realizer for EMi. 

Our semantics is "local" : we do not introduce a global variable representing the goal, 
as in continuation interpretation, in Friedman's A-translation and in Krivine's Classical 
Realizability. We interpret classical proofs "locally" and step-by-step, in order to solve a 
major problem of all computational interpretations: global illegibility, which means that, 
even for simple classical proofs, it is extremely difficult to understand how each step of the 
extracted program is related to the ideas of the proof, and what it is the particular task 
performed by each subprogram of the extracted program. The main sources of inspiration 
of this paper are works of Kleene, Hilbert, Coquand, Hayashi, Berardi and de' Liguoro and 
Avigad. 

Intuitionistic Realizability revisited. In [20] , Kleene introduced the notion of realizabil- 
ity, a formal semantics for intuitionistic arithmetic. Later, Kreisel [21] defined modified re- 
alizability, the same notion but with respect to a typed lambda calculus instead of Kleene's 
formalism of partial recursive functions. Realizability is nothing but a formal version of 
Heyting semantics for intuitionistic logic, translated into the language of arithmetic. 

Intuitively, realizing a closed arithmetical formula A means exhibiting a computer pro- 
gram - called realizer - able to calculate all relevant information about the truth of A. 
Hence, realizing a formula A V B means realizing A or realizing B, after calculating which 
one of the two is actually realized; realizing a formula 3xA(x) means computing a numeral 
n - called a witness - and realizing A(n). 

These two cases are indeed the only ones in which we have relevant information to 
calculate about the truth of the corresponding formula, and there is a decision to be made: 
realizing a formula MxA means exhibiting an algorithm which takes as input a numeral n 
and gives as output realizers of A(n); realizing a formula A A B means realizing A and 
realizing B; realizing A — )• B means providing an algorithm which takes as input realizers 
of A and gives realizers of B\ in these cases we provide no information about the formula 
we realize and we only take the inputs we will use for realizing existential or disjunctive 
formulas. Finally, realizing an atomic formula means that the formula is true: in this case, 
the realizer does nothing at all. 

Hence, intuitionistic realizability closely follows Tarski's definition of truth - the only 
difference being effectiveness: for instance, while Tarski, to assert that 3xA is true, con- 
tented himself to know that there exists some n such that A(n) is true, Kleene asked for a 
program that calculates an n such that A(n) is true. 

Intuitionistic natural deduction rules are perfectly suited to preserve realizability. In 
order to actually build realizers from intuitionistic natural deductions, it suffices to give 
realizers for the axioms. Since our goal is to interpret classical connectives using Heyting 
and Kleene interpretation of intuitionistic connectives, then a first, quite naive idea would 
be the following: if we devised realizers for Excluded Middle, we would be able to extend 
realizability to all classical arithmetic. 

Unfortunately, from the work of Turing it is well known that not every instance of 
Excluded Middle is realizable. If Txyz is Kleene's predicate, realizing \/x\/y3zTxyz V 
Mz^Txyz implies exhibiting an algorithm which for every n, m calculates whether or not 
the n-th Turing machine halts on input m: the halting problem would be decidable. Hence, 
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there is no hope of computing with effective programs all the information about the truth 
of Excluded Middle. 

However, not all is lost. A key observation is the following. Suppose we had a realizer 
O of the Excluded Middle and we made a natural deduction of a formula 3xA actually using 
Excluded Middle; then, we would be able to extract from the proof a program u, containing 
O as subprogram, able to compute the witness for 3xA. Given the effectiveness of u, after a 
finite number of steps - and more importantly, after a finite number of calls to O - u would 
yield the required witness. It is thus clear that u, to perform the calculation, would use 
only a finite piece of information about the Excluded Middle. This fundamental fact gives 
us hope: maybe there is not always necessity of fully realizing Excluded Middle, since in 
finite computations only a finite amount of information is used. If we were able to gain 
that finite information during the computation, as it is the case in the proof of Herbrand's 
Theorem, we could adapt intuitionistic realizability to Classical Logic. 

Herbrand's Theorem and the idea of "finite approximation" . (A corollary of) Herbrand's 
Theorem says that if a universal first order theory T, in a suitable language supporting 
definition by cases, proves a statement 3xP(x), then one can extract from any proof a term 
t and closed instances A\, . . . , A n of some universal formulas of T such that A\ A . . . A A n — > 
P(t) is a propositional tautology. So, even using classical logic, one can define witnesses. 
The problem is that the functions occurring in t may not be computable, because the 
language of T is allowed to contain arbitrary functions. However, given the finiteness of the 
information needed about any function used during any finite computation of t, in order 
to carry out actual calculations one would only have to find finite approximations of the 
non- computable functions involved, thus recovering effectiveness. We choose to follow this 
intuition: we will add non-computable functions to our language for realizers and exploit 
the existence of these ideal objects in order to find concrete computational solutions. 

This general idea dates back to Hilbert's e-substitution method (for a neat reformulation 
of the e- method see for example Avigad [4] ) . As noted by Ackermann [3] , the e- substitution 
method may be used to compute witnesses of provable existential statements of first order 
Peano Arithmetic. The procedure is simple: introduce Skolem functions (equivalently, e- 
terms) and correspondent quantifier free Skolem axioms in order to reduce any axiom to 
a quantifier free form; take a P^4-proof of a sentence 3xP(x) and translate it into a proof 
using as axioms only universal formulas; then apply Herbrand's theorem to the resulting 
proof, obtaining a quantifier free proof of P(t), for some term t of the extended language; 
finally, calculate a suitable finite approximation of the Skolem functions occurring in t and 
calculate from t an n such that P(n) holds. 

However, while proofs in quantifier free style are very simple combinatorial objects, 
they lose the intuitive appeal, the general concepts, the structure of high level proofs. 
Hence, it may be an impossible task to understand extracted programs. Moreover we have 
a computational syntactic method but no semantics of proofs and logical operators based 
on the idea of "finite approximation" , as the realizability interpretations are based on the 
idea of "construction". However, in the e-method, albeit only for quantifier free formulas, 
we see in action the method of intelligent learning, driven by the Skolem axioms used in 
the proofs. One of the aims of this paper is to extend this "semantics of learning" from 
atomic propositions to individuals, maps, logical connectives and quantifiers of full natural 
deduction proofs. An important contribution comes from Coquand |12j . 

Coquand's Game Semantics for Classical Arithmetic. Computing all relevant infor- 
mation about the truth of a given formula A is not always possible. In [12] and in the 
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context of game semantics, Coquand introduced a new key idea around this problem: the 
correspondence between backtracking and "learning" , a refinement of the idea of "finite ap- 
proximation" . If we cannot compute all the right information about the truth of a formula, 
maybe we could do this if we were allowed to make finitely many mistakes and to learn 
from them. 

Suppose, for instance, we have the formula Vx.ByPxy V Vy-iPxy, but we have no algo- 
rithm which, for all numeral n given as input, outputs false if Vy^Pny holds and outputs 
true if ByPny holds. Then we may describe a learning algorithm r as follows. Initially, 
for all n given as input, r outputs false. Intuitively, r is initially persuaded - following the 
principle "if I don't see, I do not believe" - that for all numeral n there is no numeral m such 
that Pnm holds. Hence, when asked for his opinion about the formula ByPny V Vy—iPny, 
r always says: ByPny is false. However, if someone - an opponent of r - to show that r 
is wrong, comes out with an m such that Pnm holds, r realizes indeed to be mistaken, 
and stores the information "Pnm is true" . Then, the next time being asked for an opinion 
about ByPny V Vy—>Pny, r will say: true. In other words, such r, after at most one "mind 
changing" , would be able to learn the correct answer to any question of the form: "which 
one among ByPny, Vy—>Pny does hold?". This is actually learning by counterexamples and 
is the key idea behind Coquand's semantics. 

Our question is now: can we formulate a realizability notion based on learning by 
counterexamples in order to extend Kreisel's interpretation to all individuals, maps and 
connectives of the sub-classical Arithmetic HA + EMi? Following Hayashi [19], in our solution 
we modify the notion of individual, in such a way that individuals change with time, and 
realizers "interact" with them. 

Hayashi's Proof Animation and Realizability. In [19] . Hayashi explains a notion of 
realizability for a sub-classical arithmetic, called limit computable mathematics. Basing 
his analysis on ideas of Gold [15], he defines a Kleene's style notion of realizability equal 
to the original one but for the fact that the notion of individual changes: the witnesses of 
existential and disjunctive formulas are calculated by a stream of guesses and "learned in 
the limit" (in the sense that the limit of the stream is a correct witness). An individual a is 
therefore a computable map a : N — > N, with a(t) representing the value of the individual 
at time t. 

For instance, how would Hayashi realize the formula Vx.ByPxy V My^Pxyl He would 
define an algorithm H as follows. Given any numeral n, H would calculate the truth 
value of My < nPny. Then the correct answer to the question: "which one among ByPny, 
Vy-iPny does hold?" is learned in the limit by computing P(n,0), P(n,l), P(n, 2),..., 
P(n, k),. . . and thus producing a stream of guesses either of the form false, false, false,. . . , 
true, true,. . . , true,. . . or of the form false, false, false, . . . , false, . . . , the first stabilizing in 
the limit to true, the second to false. Hayashi's idea is to perform a completely blind and 
exhaustive search: in such a way, the correct answer is guaranteed to be eventually learned 
(classically). Hayashi's realizers do not learn in an efficient way: in Hayashi's notion of 
realizability the only learning device is to look through all possible cases. Instead, we want 
to combine the idea of individual as limit, taken from Hayashi, with notion of learning in 
which the stream of guesses is driven by the proof itself, as in Coquand's game semantics. 
For the quantifier- free fragment, this was done by Berardi [5] and Berardi-de' Liguoro [8]. 

Realizability Based on Learning: Berardi-de' Liguoro interpretation. We explain the 
paper [8] using Popper's ideas [22] as a metaphor. According to Popper, a scientific theory 
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relies on a set of unproved - and unprovable - hypotheses and, through logic, makes predic- 
tions suitable to be falsified by experiments. If a prediction is falsified, some hypothesis is 
incorrect. In front of a counterexample to a theory's prediction, one must modify the set of 
hypotheses and build a better theory, which will be tested by experiments, and so on. Laws 
of Nature are universal statements, that cannot be verified, but are suitable to falsification. 
We may explain the link between falsifiable hypotheses and EMi. For every n, given an 
instance By.PnyVVy.^Pny of EMi (with P atomic), we may formulate an hypothesis about 
which side of the disjunction is true. If we know that Pnm is true for some m, we know 
that By.Pny is true. Otherwise we may assume My.^Pny as hypothesis, because it is a 
falsifiable hypothesis. 

We formalize the process of making hypotheses about EMi by a finite state of knowledge, 
called S, collecting the instances Pnm which we know to hold, e.g. by direct calculation. If 
we have evidence that Pnm holds for some m (that is, Pnm £ S) we know that ByPny is 
true; in the other case, we assume that My^Pny is true. So S defines a set of hypotheses on 
EMi, of the form \/y^Pny: universal falsifiable statements. Using S a realizer r may effec- 
tively decide which side of a given instance of EMi is true, at the price of making mistakes: 
to decide if Vy-iPny is true, r looks for any Pnm in the finite state S and outputs "false" 
if the research is successful, "true" otherwise. If and when from an hypothesis My^Pny we 
obtain some false conclusion —iPnm, the realizer r returns the additional knowledge: u Pnm 
is true" , to be added to S. 

Extending Berardi-de' Liguoro interpretation to HA + EMi. In our paper, we interpret 
each classical proof p of A in HA + EMi by a "learning realizer" r. r returns a "prediction" of 
the truth of this formula, based on the information in S, and some additional knowledge in 
the case the prediction is effectively falsified. For example, in front of a formula Bx.A A B, 
a realizer r predicts that A(n) A B(n) is true for some numeral n (and since n depends on 
s, in our model we change the notion of individual, interpreting "numbers" as computable 
maps from the set of bases of knowledge to N). Then r predicts, say, that B(n) is true, 
and so on, until r arrives at some atomic formula, say -^Pnm. Either Pnm is actually true, 
or r is able to effectively find one or more flawed hypothesis \/x.-iQin±x, . . . , Vx.-iQfcnfcX 
among the hypotheses used to predict that Pnm is true, and for each flawed hypothesis 
one counterexample Q\nim\, . . . , Qk n k m k- I n this case, r requires to enlarge our state of 
knowledge S by including the information "Q\n\m\ is true", . . . , u Qknk m k 1S true". 

Our Interactive Realizability differs from Intuitionistic Realizability in the notion of 
individual (the value of an individual may depend on our knowledge state), and in the 
realizability relation for the atomic case. In our interpretation, to realize an atomic formula 
does not mean that the formula is true, but that the realizer requires to extend our state of 
knowledge S if the formula is not true. The realizer is thought as a learning device. Each 
extension of S may change the value of the individuals which are parameters of the atomic 
formula, and therefore may make the atomic formula false again. Then the realizer requires 
to extend S again, and so forth. The convergence of this "interaction" between a realizer 
and a group of individuals follows by Avigad's fixed point thm. [3] (a constructive proof 
may be found in [5]), and it is the analogue of the termination of Hilbert's e-substitution 
method. 

Why the Arithmetic HA + EMi instead of considering the full Peano Arithmetic? We have 
two main reasons. First, we observe that EMi enjoys a very good property: the information 
about its truth can be computed in the limit, in the sense of Gold [15], as we saw en 
passant when discussing Hayashi's realizability. This implies that witnesses for existential 
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and disjunctive statements too can be learned in the limit, as shown in Hayashi [19J. In a 
forthcoming paper we show that realizers which we will be able to extract from proofs have 
a straightforward interpretation as winning strategies in 1-Backtracking games [7j, which 
are the most natural and simple instances of Coquand's style games. Secondly, a great deal 
of mathematical theorems are proved by using EMi alone ([I], [6]). 

Plan of the Paper. The paper is organized as follows. In £}2]we define the term calculus 
in which our realizers will be written: a version of Godel's system T, extended with some 
syntactic sugar, in order to represent bases of knowledge (which we shall call states) and 
to manipulate them. Then we prove a convergence property for this calculus (as in Avigad 
[1] or in [5]). In $31 we introduce the notion of realizability and prove our Main Theorem, 
the Adequacy Theorem: "if a closed arithmetical formula is provable in HA + EMi, then it is 
realizable" . In ^5] we conclude the discussion about our notion of realizability by comparing 
it with other notions of realizability for classical logic, then we consider some possible future 
work. 

2. The Term Calculus 

In this section we formalize the intuition of "learning realizer" we discussed in the 
introduction. 

We associate to any instance 3yPxy V My^Pxy of EMi (Excluded Middle restricted to 
Sj'-formulas) two functions xp an d <pp- The function \P takes a knowledge state S, a 
numeral n, and it returns a guess for the truth value of By.Pny. When this guess is "true" 
the function <pp returns a witness m of By.Pny. The guess for the truth value of By.Pny is 
computed w.r.t. the knowledge state S, and it may be wrong. For each constant s denoting 
some knowledge state S, the function Ax : N.xp(s,x) is some "approximation" of an ideal 
map Ax : N.Xp(x), the oracle returning the truth value of By.Pxy. In the same way, the 
function Ax : N.(/>p(s,x) is some "approximation" of an ideal map Ax : N.3>p(x), the Skolem 
map for By.Pxy, returning some y such that Pxy if any, and otherwise. The Skolem 
axioms effectively used by a given proof take the place of a set of experiments testing the 
correctness of the predictions made by ipp(s,x),xp(s,x) about Xp(x),$p(x) (we do not 
check the correctness of (pp,xp in an exhaustive way, but only on the values required by 
the Skolem axioms used by a proof). 

Our Term Calculus is an extension of Godel's system T ■ For a complete definition of T 
we refer to Girard [H]. T is simply typed A-calculus, with atomic types N (representing the 
set N of natural numbers) and Bool (representing the set B = {True, False} of booleans), 
product types TxU and arrows types T — > U, and pairs (., .), projections ttq, tt±, conditional 
if t and primitive recursion Rt in all types, and the usual reduction rules (/3), (it), (if), (R) 
for A, (., .}, if t, Rt- From now on, if t, u are terms of T with t = u we denote provable 
equality in T . If k E N, the numeral denoting k is the closed normal term k = S (0) of type 
N. We denote numerals in T by n, m, and natural numbers with i,j, k,h, . . . £ N. All closed 
normal terms of type N are a numeral. We denote with True, False : Bool the boolean 
constants of T ■ Any closed normal term of type Bool in T is True or False. 

We introduce a notation for ternary projections: if T = A x (B x C), with Po,P\,P2 we 
respectively denote the terms ttq, Ax : T.ttq(tti(x)), Ax : T.7ri(7ri(x)). If u = (uq, (u\,U2)) : 
T, then piU = Ui in T for i = 0, 1,2. We abbreviate (uo, (u\,U2)) : T with (uq,u\,U2) : T. 
We formalize the idea of "finite information about EMi" by the notion of state of knowledge. 
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Definition 2.1 (States of Knowledge and Consistent Union). 

(1) A fc-ary predicate of 7" is any closed normal term P : N fc — )• Bool of 7". 

(2) An atom is any triple (P, ft, m), where P is a (k + l)-ary predicate, and n, m are {k + 1) 
numerals, and Pnm = True in T. 

(3) Two atoms (P,ft,m), (P',n',m') are consistent if P = P' and n = n 1 va. T imply 
m = m! . 

(4) A state of knowledge, shortly a state, is any finite set S of pairwise consistent atoms. 

(5) Two states Si, S*2 are consistent if Si U S2 is a state. 

(6) § is the set of all states of knowledge. 

(7) The consistent union S1US2 of Si, S2 G S is Si U S2 € S minus all atoms of S2 which 
are inconsistent with some atom of Si. 

We think of an atom (P, ft, m) as the code of a witness for 3y.P(n, y). Consistency condition 
allows at most one witness for each 3y.P(n, y) in each knowledge state S. Two states Si, S2 
are consistent if and only if each atom of Si is consistent with each atom of S2 . 

S1US2 is an non-commutative operation: whenever an atom of Si and an atom of S2 
are inconsistent, we arbitrarily keep the atom of Si and we reject the atom of S2, therefore 
for some Si , S2 we have S1US2 7^ S2US1 . U is a "learning strategy" , a way of selecting a 
consistent subset of Si U S2 . It is immediate to show that U is an associative operation on 
the set of consistent states, with neutral element 0, with upper bound S1US2, and returning 
a non-empty state whenever Si U S2 is non-empty. 

Lemma 2.2. Assume i G N and Si, . . . , Sj G S. 

(1) SiU...USi CSiU...USi 

(2) SiU ...USi = % implies S x = . . . = Si = 0. 

In fact, the whole realizability Semantics is a Monad [10]. In [TO], it is proved that 
our realizability Semantics is parametric with respect to the definition we choose for U. 
Any associative operation U, with neutral element and satisfying the two properties of 
Lemma r2.21 defines a different but sound realizability Semantics, corresponding to a different 
"learning strategy". An immediate consequence of Lemma 12.21 is: 

Lemma 2.3. Assume S, Si,S2 G S. 

(1) If S is consistent with Si,S2, then S is consistent with SilAS2- 

(2) If S is disjoint with Si,S2, then S is disjoint with SiUS2- 

For each state of knowledge S we assume having a unique constant s = S denoting 
it: for instance, is a state constant denoting the empty state. We define with 7s = 
T + S + {S|S € §} the extension of T with one atomic type S denoting S, and a constant 
s = S : S for each S G S, and no new reduction rule. We denote states by S, S',. . . and 
state constants by s, s' , . . .. Any closed normal form of type N, Bool, S in 7s is, respectively, 
some numeral n, some boolean True, False, some state constant s. Computation on states 
will be defined by some suitable set of algebraic reduction rules we call "functional" . 

Definition 2.4. (Functional set of rules) Let C be any set of constants, each one of some 
type A\ —■ . . . — >• A n —> A, for some A±, . . . ,A n , A G {Bool,N,S}. We say that TZ is a 
functional set of reduction rules for CifTZ consists, for all c G C and all ai : A±, . . . , a n : A n 
closed normal terms of 7s, of exactly one rule cai . . . a n *— Y a, for some closed normal term 
a : A of 7s. 
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Theorem 2.5. Assume that 1Z is a functional set of reduction rules for C (def. \2.4\ ). Then 
7s + C + 1Z enjoys strong normalization and weak-Church-Rosser (uniqueness of normal 
forms) for all closed terms of atomic types. 

Proof. (Sketch) For strong normalization, see |11| (the constants s : S and c G C are 
trivially strongly computable). For weak Church- Rosser property, we start from the fact 
that there is the canonical set-theoretical model Ai of 7s + C + 1Z. The interpretation of 
Bool, N, S in Ai consists of all closed normal form of these types. Arrows and pairs are 
interpreted set-theoretically. Each constant c G C is interpreted by some map f c , defined 
by / c (ai, . . . , a n ) = a for all reduction rules (coi...a n i-> a) G 1Z. Assume u,v : A are 
closed normal term, A = Bool, N, or S is an atomic type, and u, v are equal in 7s + C + 1Z, 
in order to prove that u, v are the same term, u, v are equal in A\ because A4 is a model of 
7s + C + 1Z. By induction on w we prove that if w is a closed normal form of atomic type 
T + C + 1Z, then iu is a numeral, or True, False, or a state constant, and therefore w is 
interpreted by itself in Ai. From u, v equal in M. we conclude that u,v are the same term 
of 7~s + C + iz. □ 

We define two extensions of 7s: an extension 7ciass with symbols denoting the non- 
computable maps Xp, <l?p and no computable reduction rules, another extension TLcam! 
with the computable approximations %p, (j)p of Xp, $p, and a computable set of reduction 
rules. We use the elements of 7ciass to represent non-computable realizers, and the elements 
of TLcarn to represent a computable "approximation" of a realizer. In the next definition, 
we denote terms of type S by p, p', — 

Definition 2.6. Assume P : N fc+1 — > Bool is a k + 1-ary predicate of T. We introduce the 
following constants: 

(1) xp '■ S ->■ N fc ->• Bool and ip P : S ->■ N fe ->• N. 

(2) A P : N fc ->■ Bool and $ P : N fe -> N. 

(3) y : S ->■ S -)■ S. 

(4) Addp : N fc+1 -^ S and add P : S -> N fc+1 -4- S. 
We denote ypi/32 with pi y p2- 

(1) S s is the set of all constants xp-> <Pp> ^> addp. 

(2) H is the set of all constants Xp, $p, y, Addp. 

(3) 7ciass = 7s + S. 

(4) A term t G 7ciass has state if it has no state constant different from 0. 

Let t = t\ . . . tk- We interpret XP s t and cppst respectively as a "guess" for the values 
of the oracle and the Skolem map Xp and <^p for By.Pty, guess computed w.r.t. the 
knowledge state denoted by the constant s. There is no set of computable reduction rules 
for the constants <3?p, Xp G S, and therefore no set of computable reduction rules for 7ciass- 
If pi,p2 denotes the states Si, S^ S S, we interpret p\ y p2 as denoting the consistent 
union S1US2 of S\,S2- Addp denotes the map constantly equal to the empty state 0. 
addpSnm denotes the empty state if we cannot add the atom (P, n, m) to S, either 
because (P, n, ml) G S for some numeral m! , or because Pnm = False. addpSnm denotes 
the state {(P,n,m)} otherwise. We define a system TLoarn with reduction rules over E s by 
a functional reduction set 7£s- 

Definition 2.7. (The System Tloam) Let s,s\,S2 be state constants denoting the states 
S, Si, S2. Let (P, n, m) be an atom. 7^-s is the following functional set of reduction rules for 

^s: 
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(1) If (P, n, m) € S, then XP sn " ^ True and ippsn \- > m, else Yp sn * ^ False and tppsn H > 0. 

(2) si y S 2 !-». 5i^,g 2 

(3) addpsnm h )• if either {P,n,m') € <S for some numeral m' or Pnm = False, and 
addpsnm i— > {(P,n,m)} otherwise. 

We define 7I carn = Ts + H s + TZ S . 

Remark. TLcam is nothing but 7s with some "syntactic sugar". By Theorem [231 TLcam 
is strongly normalizing and has the weak Church-Rosser property for closed term of atomic 
types. TLcam satisfies a Normal Form Property. 

Lemma 2.8 (Normal Form Property for TLcam) • Assume A is either an atomic type or a 
product type. Then any closed normal term t £ Theam of type A is: a numeral n : N, or a 
boolean True, False : Bool, or a state constant s : S, or a pair (u,v) : B x C. 

Proof. (Sketch) By induction over t. For some v, either t is (Xx.u)(v), or t is (u, w)(v), or t is 
x{v) for some variable x, or t is c(v) for some constant c, and either c = 0, S, True, False, s, 
Rt, if t,7Tj is some constant of 7s, or c G H s . If t = (Xx.u)(v), then t has an arrow type 
if v = 0, while t is not normal if v ^ 0. If t = (u,w)(v), then v = and we are done. If 
£ = x(iT) then £ is not closed. The only case left is t = c(u) : A. A is not an arrow type, 
therefore all arguments of c are in u. If t = we are done, if t = S(u) we apply the induction 
hypothesis, if t = True, False : Bool or t = s : S or t = (u, v) we are done. Otherwise 
either t = Rr{n, f,a)t, if r(6, a\, a,2)t,TTi(v)t, or t = Xp( u -> $) : N, or £ = (pp(u,w) : N, or 
t = W(u\,U2) : S, or t = addp(u, w) : S. The proper subterms n,Wi, . . . ,Wk : N, b : Bool, 
v : Ax B, u,u±,U2 '■ S of t have atomic or product type and are closed normal. By induction 
hypothesis they are, respectively, a numeral, a boolean, a pair, a state constant. In all cases, 
t is not normal. □ 

Let t, t' & TLcam be two closed terms of type S. We abbreviate "£, t' denotes two states 
which are consistent and disjoint" by: t,t' are consistent and disjoint. 0, s are consistent 
and disjoint for every state constant s. The maps denoted by U=U , addp preserve the relation: 
"to be consistent and disjoint". 

Lemma 2.9. Assume s,s\,S2 are state constants and (P,n,m) is an atom. 

(1) s, (addpsnm) are consistent and disjoint. 

(2) Assume s,s\ are consistent and disjoint, and s,S2 are consistent and disjoint. Then 
s, si iyj S2 are consistent and disjoint. 

Proof. 

(1) Assume s denotes the state S. If addpsnm denotes the empty state the thesis is 
immediate. Otherwise addpsnm denotes {(P,n,m)} and (P,n,mf) (jL S for all numeral 
m'. Then {(P,n,m)} is consistent and disjoint with S. 

(2) By Lemma [23J □ 

Each (in general, non-computable) term t G 7ciass is associated to a set {t[s] |s is a 
state constant} C TLcam of computable terms we call its "approximations", one for each 
state constant s. 

Definition 2.10. Assume t € 7ciass and s is a state constant. We call "approximation of t 
at state s" the term t[s] of TLcam obtained from t by replacing each constant Xp with xp s -> 
each constant <i>p with tpps, each constant Addp with addps. 
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We interpret any t[s] G TL&axn as a learning process evaluated w.r.t. the information 
taken from a state constant s (the same s for the whole term). 

Assume t G 7ci aS s is closed, t : S and s is a state constant. Then t[s] is a closed term 
of 7Learn> and its normal form, by the Normal Form Property 12.81 is some state constant s'. 
We conclude t[s] = s' in 7Leam- We prove that s, s' are consistent and disjoint. 

Lemma 2.11. Assume s is a state constant, t G Tciass, t : S is closed, and all state constants 
in t are consistent and disjoint with s. 

(1) If t[s] reduces to t'[s], then all state constants in t' are consistent and disjoint with s. 

(2) s,t[s] are consistent and disjoint. 

(3) If all state constants in u are 0, then s,u[s] are consistent and disjoint. 

Proof. 

(1) It is enough to consider a one-step reduction. Suppose that t[s] reduces to t'[s] by 
contraction of a redex r of t[s]. If r is (Xxu)t or RtuvS(w) or ifr(6, ai, 0,2) or 7rj(vi, V2) 
or xps™: or (fpsn, then its contractum r' does not contain any new state constant; hence, 
all state constants in t' are consistent and disjoint with s. If r is s% W S2 or addpsnm, 
then both s, si and s, S2 are consistent and disjoint state constants by hypothesis on t; 
therefore, by Lemma 12. 9| in both cases s and the contraction of r are consistent and 
disjoint; so all state constants in t' are consistent and disjoint with s. 

(2) Every reduct of t[s] is t'[s] for some t' G 7ci ass - If t[s] reduces to a normal form t'[s] = s', 
then the only possibility is t' = s'. By the previous point 1, we conclude that s' is 
consistent and disjoint with s. 

(3) By the previous point 2, and the fact that the only state constant in u is consistent 
and disjoint with any s. □ 

We introduce now a notion of convergence for families of terms {i[sj]}i S N C 7L 0ar „, defined by 
some t G 7ci ass and indexed over a set of state constants {sj}j G n. Informally, "t convergent" 
means that t[s] eventually stops changing when the knowledge state s increases. If s, s 1 are 
state constants denoting S, S' € S, we write s < s' for S C S' . We say that a sequence 
{•SjjjgN of state constants is a weakly increasing chain of states (is w.i. for short), if Si < Sj+i 
for all i € N. 

Definition 2.12. (Convergence). Assume that {sjjigN is a w.i. sequence of state constants, 
and u,v G 7ci ass - 

(1) u converges in -j>j} iGN if 3i G N.Vj > i.u[sj] = u[sj in 7I carn . 

(2) u converges if u converges in every w.i. sequence of state constants. 

Remark that if u is convergent, we do not ask that u is convergent to the same value on 
all w.i. chain of states. The value learned by u may depend on the information contained 
in the particular chain of state constants by which u gets the knowledge. The chain of 
states, in turn, is selected by the particular definition we use for the "learning strategy" U. 
Different "learning strategies" may learn different values. 

Theorem 2.13 (Stability Theorem). Assume t G Tciass is a closed term of atomic type A 
(A G {Bool,N, S}). Then t is convergent. 

Proof. (Classical). Assume S is any consistent and possibly infinite set of atoms. We define 
some (in general, not computable) functional reduction set 7Z(S) for the set 3 of constants 
and for 7ci ass - The reductions for Xp, $p, Addp are those for xp-, 4>Pi addp in TLeam' 
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(1) If {P,n,m) G S, then (X P n i-> True), ($pn \-> m) € Tl(S), else 
(X P n ^ False), ($ P n ^ 0) G ft(S) 

(2) Addpnm i— > if either (P, n, m!) G S for some numeral m' or Pnm = False, and 
Addpnm i— >■ {(P,n,m)} o.w.. 

and the reduction for W in TZ(S) is the reduction for W in TLam- By theorem 12.51 Tciass + 
'R-(S) is strongly normalizing and weak-CR for all closed terms of atomic type, for any 
consistent set of atoms S. For the rest of the proof, let {sj}j £ n be a w.i. chain of state 
constants. Assume t G 7ciass is a closed term of atomic type A. Claim. For any state 
constant s, the map u t— > u[s] is a bijection from the reduction tree of t in 7ciass + Tl{ s ) 
to the reduction tree of t[s] in TLeaxn- Proof of the Claim. By induction over the reduction 
tree of t[s]. Every reduction /3,7r, if t,Rt, W over t[s] may be obtained from the same 
reduction over t. All occurrences of xp,y?p,addp m the reduction tree of t[s] are of the 
form xps,^>ps,addps, therefore every reduction over xp,<£p,addp may be obtained from 
the corresponding reduction over Xp, <3?p, Addp. 

Assume a is the (unique, by weak-CR) normal form of t in 7ciass + TZ{s). By the 
Claim, a[s] is the normal form of t[s] in 7Leam- Since a is normal in 7ciass + T^(s), there 
is no Xp,&p,Addp in a. Thus a and a[s] are the same term: t and i[s] have the same 
normal form respectively in 7ciass + T^(s) and in TLcam- Let {sj}j G n be a given sequence 
of state constants. Define S^ = Ui^Si, where Si is the state denoted by Sj. By strong 
normalization, the reduction tree of t in 7ci aS s + T^(Suj) is finite. Therefore in this reduction 
tree are used only finitely many reduction rules from 1Z(S W ), and for some numeral n it is 
equal to the reduction tree of t in 7ciass + T^(s n ), and in 7cia SS + ^(s m ) for all m > n. We 
deduce that for all m > n the normal forms of t in 7ci aS s + K(s m ) are the same. Thus, the 
normal form in 7Leam of all t[s m ] with m > n are the same, as we wished to show. fj 



Remark 2.14. The idea of the proof of theorem 12.131 corresponds exactly to the intuition 
of the introduction. During any computation, the oracles Xp and <3?p are consulted a finite 
number of times and hence asked for a finite number of values. When our state of knowledge 
is great enough, we can substitute the oracles with their approximation yps and ipps for 
some state constant s, and we will obtain the same oracle values and hence the same results. 

The proof, though non constructive, is short and well explains why the result is true. 
However, provided we replace the notion of convergence used in this paper with the intu- 
itionistic notion introduced in [5], we are able to reformulate and prove theorem 12.131 in a 
purely intuitionistic way, achieving thus a constructive description of learning in HA + EMi . 
Being the intuitionistic proof way more elaborated and less intuitive than the present one 
and connected with other foundationally interesting results, it will be the subject of a next 
paper. 

Our proof of convergence follows the pattern of Avigad's one in [3]. A closed term 
t G 7ciass of atomic type and in the constant c±, . . . , c n G H, may be seen as a functional Ft 
which maps functions /i, . . . , f n of the same type of ci, . . . c n into an object of atomic type: 
Ft(fi, ■ ■ ■ , fn) is defined as the normal form of t in 7ciass + 7Z, where 1Z = {c^ai . . . a n \-> 
o | fi(a>i, ■ ■ ■ , a n ) = a and i G {1, . . . , n}}. F t is continuous in the sense of Avigad. Moreover, 
since Xp and Addp have a set-theoretical definition in terms of ^p, we may assume Ft 
depends only on the functions which define in 1Z the reduction rules for $p x , . . . $p n - Then, 
if t is of type S, it is not difficult to see that F t represents an update procedure with respect 
to any of its argument. The fact that Ft is an update procedure implies convergence for t 
and the fixed point property of theorem 12.151 
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Assume that s is a state constant and t G 7ciass any closed term of type S of state 
(i.e., without state constants different from 0). Denote by r the map :§—>■§ interpreting 
s i-)- t[s]. t is defined by t(S) = S' if and only if t[S] = S_' in TLcarn- By Lemma [2.111 
S,t(S) are consistent and disjoint. In particular, f(S) = SUt(S) defines a map / : S — > S. 
By Theorem 12.131 if {5j}j GN is any w.i. sequence of states, then 3i.Vj > i.r(Sj) = r(5j). 

As last result of this section, we prove that if we start from any state S, and we 
repeatedly apply /:§—>§, eventually we reach a state S' = f n (S) such that f(S') = S' 
and t(S) = 0. We interpret this result by saying that / is a "learning process" adding the 
knowledge computed by the map r, and / eventually stops extending the knowledge. 

Theorem 2.15 (Fixed Point Property). Let t : S be a closed term of Tciass of state 0, and 
s = S. Define t(S) = S' if t\S] = S', and f(S) = S U t(S). 

(1) There are h G N, S' G § such that S' = f h (S) D S, f(S') = S' and t(S') = 0. 

(2) We may effectively find a state constant s' > s such that t[s'} = 0. 

Proof. 

(1) f°(S),f 1 {S),f 2 (S), ... is a w.i. chain of states because f(S') 2 S f for all S f G S. By 
theorem 12.131 the map r : S —?■ §>, interpreting the map s h-> t[s], converges over this 
chain: there exists k G N such that for every j > k, t(P(S)) = r(/ fc (s)). By definition 
of / and the choice of k: 

fk+ 2 {s) = fk+ l {s) u r (/fc+l( S )) = (/*(£) U T(f k (S))) U T(f k (S)) = 

= f k (S)Ur(f k (S)) = f k+1 (S) 

Choose S' = f k+1 (S). By the line above, we have S' > S and f(S') = S', therefore 
t(S') C f(S') = S'. From S',t(S') disjoint we conclude r(S') = 0. 

(2) By the previous point and t\S_'} = if and only if t(S') = 0. D 

3. An Interactive Learning-Based Notion of Realizability 

In this section we introduce the notion of realizability for HA + EMi, Heyting Arithmetic 
plus Excluded Middle on X^-formulas, then we prove our Main Theorem, the Adequacy 
Theorem: "if a closed arithmetical formula is provable in HA + EMi, then it is realizable". 

We first define the formal system HA+EMi, from now on "Extended EMi Arithmetic". We 
represent atomic predicates of HA + EMi with (in general, non-computable) closed terms of 
Tciass of type Bool. Terms of HA + EMi may include function symbols Xp, $p denoting non- 
computable functions: oracles and Skolem maps for S^-formulas 3x.Pxn, with P predicate 
of T . We assume having in T some terms =>Booi : Bool, Bool — > Bool, -i B ooi : Bool — >• 
Bool, . . ., implementing boolean connectives. If ti, . . . ,t n ,t G T have type Bool and are 
made from free variables all of type Bool, using boolean connectives, we say that t is a 
tautological consequence of t\, . . . , t n in T (a tautology if n = 0) if all boolean assignments 
making ti,...,t n equal to True in T also make t equal to True in T ■ 

Definition 3.1. (Extended EMi Intuitionistic Arithmetic: HA + EMi) The language £ciass of 
HA + EMi is defined as follows. 

(1) The terms of £ciass are all t G 7ciass with state 0, such that t : N and FV(t) C 
{xi, . . . ,x™} for some xi, . . . ,x n . 

(2) The atomic formulas of £ciass are all Qt\ . . . t n G 7ci aS s) f° r some Q : N n — > Bool closed 
term of Tciass of state 0, and some terms ti,...,t n of £ciass- 
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(3) The formulas of £ciass are built from atomic formulas of £ciass by the connectives 

V, A, — > V, 3 as usual. 

A formula of HA is a formula of HA + EMi in which all predicates and terms are terms of 

r. 

Deduction rules for HA + EMi are as in van Dalen [13] . with: (i) an axiom schema for 
EMi; (ii) the induction rule; (Hi ) as Post rules: all axioms of equality and ordering on N, all 
equational axioms of 7", and one schema for each tautological consequences of 1~. (iv) the 
axiom schemas for oracles: P(t, t) =>Booi ^pt*and for Skolem maps: Xpt =>Booi P(t, ($P*))) 
for any predicate P of 7". 

We denote with _L the atomic formula False and will sometimes write a generic atomic 
formula as P(t\, . . . , t n ) rather than in the form Pt\ . . . t n . Finally, since any arithmetical 
formula has only variables of type N, we shall freely omit their types, writing for instance 
\/x.A in place of Vx .A. Post rules cover many rules with atomic assumptions and conclusion 
as we find useful, for example, the rule: "if f(z) < then f(z) = 0". 

We defined =>Booi : Bool, Bool — > Bool as a term implementing implication, therefore, 
to be accurate, the axiom P(t\, ... ,t n ,t) =>booi Xpt\ . . . t n is not an implication between 
two atomic formulas, but it is equal to the single atomic formula Qt\ . . . t n t, where 

Q = Xxl . . . Ax" +1 . =^booi (Pxi . . . x n x n+1 )(X P xi . . . X n+ l) 

Similarly, -, Booi-P(£, t) will denote a single atomic formula. Any atomic formula A of £ciass 
is a boolean term of 7ciass> therefore for any state constant s we may form the "finite 
approximation" A[s] : Bool, A[s] G Thcam of A. In A[s] we replace all oracles Xp and all 
Skolem maps $p we have in A by their finite approximation xp s )0p s ) computed with 
respect to the state constant s. We denote with £ Learn the set of all expressions A[s] with 
A € £ciass and s a state constant. All A[s] £ £ Lea rn may be interpreted by first order 
arithmetical formulas having all closed atomic subformulas decidable. 

Using the metaphor explained in the introduction, we use a set of falsifiable hypotheses 
determined by s to predict a computable truth value A[s] : Bool in TLcam for an atomic 
formula A £ £ciass that we cannot effectively evaluate. Our definition of realizability pro- 
vides a formal semantics for the Extended Intuitionistic Arithmetic HA + EMi , and therefore 
also for the more usual language of Arithmetic HA, in which all functions represent recursive 
maps. 

Definition 3.2. (Types for realizers) For each arithmetical formula A we define a type \A\ 
of T by induction on A: \P(h, . . . ,t n )\ = S, \AAB\ = \A\x \B\, \AVB\ =Boolx(|A| x \B\), 
\A->B\ = \A\ -> \B\, \VxA\ = N -)■ \A\, \3xA\ = N x |A| 

We define the realizability relation t III — A, where t E 7cia SS) A G £ciass, t has state 
and t : \A\. The realizer denotes a non-computable map t, and is associated to a family 
{i[s]|s state constant} of one computable map t[s] for each s, realizing the approximation 
A[s] € >C Loam of the formula A. We interpret the set of Excluded Middle instances and 
Skolem axioms effectively used by a given proof as a set of experiments checking the as- 
sumptions we have in s about Skolem maps and oracles. If all experiments succeed, the 
realizer provides a "construction" for A; if some experiment fails, the realizer provides some 
new knowledge obtained from the failure. 

We first define t \\- s A, the realizability relation for the "approximations" t € "/Learn and 
A £ ^Loam, w.r.t. any state constant s, then we define t' 1 1 1 — A' for t' G 7ciass of state 
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and any closed A' £ £ciass- For any types T, U, V, let PO1PI1P2 denote the three projections 
from T x(U xV). 

Definition 3.3. Let s be the constant denoting a state S £ §. Assume t £ Team and 
A £ TLoarn are OI " the form £ = i'[s], A = -A'[s] for some closed t' £ 7ciass of state and some 
closed A' £ £ C i ass . We define t \\- s A for any state constant s by induction on A. 

(1) £ Ihj P(ti, . . . , t n ) if and only if t = in TLam implies P(ti, . . . ,t n ) = True 

(2) t\\- s AAB if and only if 7r £ lh s A and 7Ti£ lh s B 

(3) £ lh s ^4 V B if and only if: either pot = True in 7i\ cavn and pit lh s A, or po^ = False and 
p 2 t \h s B 

(4) t \\- s A — >• i? if and only if for all u, if u lh s A, then tu\\- s B 

(5) £ lh s \/xA if and only if for all numerals n, tn \\- s A[n/x] 

(6) £ \\~ s 3xA if and only if for some numeral n ir^t = n in T^am an d nit \\~ s A\n/x\ 
Assume t' £ 7ciass is a closed term of state 0, A' £ C Ciass is a closed formula, and t' : \A'\. 
We define 

(1) t' \\\- a A' if and only if t'[s] lh s A'[s] 

(2) t' 1 1 1 — A' if and only if t' \\\- s A' for all state constants s. 

The realizability relation is compatible with equality in Thc&m'- 
Lemma 3.4. If t\ \\- s A[u\/x\, t\ = ti and u± = ui in Theam, then £2 H~s A [1*2/3;] 
Proof. By straightforward induction on A. □ 

By unfolding the definition of t \\\- s A, we may obtain a direct characterization of the 
realizability relation for terms t of 7ciass) bypassing the reference to the relation lh s over 
"approximations" of terms and formulas of Cq\ &ss . The only clause for t llh s A which is 
(slightly) different from the clause for t lh s A is the clause for atomic formulas. We write 
the characterization of I II — explicitly because we refer to it in the next discussion. 

Lemma 3.5 (Realizability). Assume s is a state constant, t £ Tciass is a closed term, 
A £ Cciass is 0, closed formula, and t : \A\. Let t = ti, ■ ■ ■ , t n : N. 

(1) t \\\- s P(t) if and only ift[s] = in TLa™ implies P(t)[s] = True 

(2) t llh s A A B if and only if not llh s A and -Kit \\\- s B 

(3) t III— s A\/ B if and only if either pot[s] = True in Theam andp\t \\r s A, or p§t\s\ = False 
in Theam and p 2 t \\\- s B 

(4) t III— s A — ^ B if and only if for all u, if u \\\- s A, then tu \\\- s B 

(5) t llh s VxA if and only if for all numerals n, tn \\V S A[n/x] 

(6) t III— s 3x^4 i/ and on/y for some numeral n, 7Toi[s] = n in Theam and n\t \\\- s A[n/x\[s\ 

Proof. By definition unfolding. □ 

The characterizations of III— shows that the definition of III— formalizes all the idea we 
sketched in the introduction. A realizer is a term t of 7ciass 5 possibly containing the non- 
computable functions Xp, $p; if such functions were computable, t would be an intuitionis- 
tic realizer. Since in general t is not computable, we calculate its approximation t[s] at state 
s, which is a term of 7Lcarn> and we require it to satisfy the indexed- by-state realizability 
clauses. Realizers of disjunctions and existential statements provide a witness, which is an 
individual depending on an actual state of knowledge, representing all the hypotheses used 
to approximate the non-computable. The actual behavior of a realizer depends upon the 
current state of knowledge. The state is used only when there is relevant information about 
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the truth of a given formula to be computed: the truth value P(t\, ■ ■ ■ , t n )[s] of an atomic 
formula and the disjunctive witness po^H an d the existential witness 7Tou[s] are computed 
w.r.t. the constant state s. A realizer t of A V B uses s to predict which one between A 
and B is realizable (if po^[ s ] = True then A is realizable, and if po^H = False then B is 
realizable). A realizer u of 3xA uses s to predict that vron[s] equals an n, some witness for 
3xA (i.e. that j4[n/x] is realizable). These predictions need not be always correct; hence, 
it is possible that a realized atomic formula is actually false; we may have t \\\- s P and 
P[s] = False in "/Learn- If an atomic formula, although predicted to be true, is indeed false, 
then we have encountered a counterexample and so our theory is wrong, our approximation 
still inadequate; in this case, t[s] 7^ by definition of t llh s P, and the atomic realizer t takes 
s and extends it to a larger state s', union of s and t[s]. That is to say: if something goes 
wrong, we must learn from our mistakes. The point is that after every learning, the actual 
state of knowledge grows, and if we ask to the same realizer new predictions, we will obtain 
"better" answers. 

Indeed, we can say more about this last point. Suppose for instance that t 1 1 1 — A V B 
and let {sj}j G n be a w.i. sequence. Then, since t £ |Bool| x \A\ x \B\, then p$t : Bool is a 
closed term of 7ciass, converging in {si}j<=N to a boolean; thus the sequence of predictions 
Po^[sn] eventually stabilizes, and hence a witness is eventually learned in the limit. 

In the atomic case, in order to have t \\\- s P(t\,. . . ,t n ), we require that if t[s] = 0, 
then P(t\, . . . , t n )[s] = True in /Learn- That is to say: if t has no new information to add 
to s, then t must assure the truth of P{t±, . . . ,t n ) w.r.t. s. By the Fixed Point Property 
(theorem 12. 15f) . when t : S is closed, there is plenty of state constants s such that t[s] = 0; 
hence search for truth will be for us computation of a fixed point, driven by the excluded- 
middle instances and the Skolem axioms used by the proof, rather than exhaustive search 
for counterexamples. 

Example 3.6. The most remarkable feature of our Realizability Semantics is the existence 
of a Ep realizer for EMi . Assume that P is a predicate of T and define Ep as 

Xa N {X P a, (<£ P a, 0), An N Add P an)) 

Proposition 3.7. (Realizer E P o/EMj E P I II — W. By P(x,y) V\/y^ Bool P(x,y). 

Proof. Let m be a vector of numerals and let s = S be a state constant denoting S € S. 
Epm[s] is equal to 

(xpsrh, (ippsm, 0), An N addpsmn) 
and we want to prove that 

E P m[s] \\- s By P(rh,y) V\/y^ B ooiP(rh,y) 

We have poEprh[s] = xp-sw, in TLeam- Assume XP sr ^ = True. Then (P,m,n) G S for some 
numeral n such that P(rh, n) = True, and we have to prove 

piEpm[s\ \V a By P(rh,y) 

By definition unfolding, piEpm[s] = {(ppsm,®) = (by definition of (pp(s,rh)) (n,0), hence, 
iro(piEpm)[s] = 7i"o((n,0)) = n and pi(iriEpm)[s] \\- a P(rh,n) because P(rh,n) = True. 
We conclude p\Epm[s\ \\- s By P(fh,y). Now assume poEpfh[s] = XP sr ^ = False. Then 
{P, fh, n') £ S for all numerals n'. We have to prove 

P2Epfh[s] = Xn addpsmn lh a Vy-i B ooi-P('Tij y) 
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that is that, given any numeral n, 

addpsfhn \\- a ->booi P(m,n) 

By the definition of realizer in this case, we have to assume that addpsfhn = 0, in order to 
prove that -■booi-P^, n )[s] = True. The substitution (.)[s] has an empty effect over P(m, n), 
therefore we have to prove that _, Booi-P(^ 5 n) = True, that is, that P(fh, n) = False. 
Assume for contradiction that P(fh, n) = True. We already proved that (P, fh, n') £ S, 
for all numeral n'\ from this and P(fn,n) = True we deduce addpsmra = {(P,m,n)}, 
contradiction. □ 

Ep works according to the ideas we sketched in the introduction. It uses \P to make 
predictions about which one between 3y P(fh,y) and Vy-iBooi-P(wj y) is true. \P-> i n turn, 
relies on the constant s denoting the actual state to make its own prediction. If XP sm = 
False, given any n, ->sooiP(Tn, n) is predicted to be true; if it is not the case, we have a 
counterexample and Addp requires to extend the state with (P,fh,n). On the contrary, 
if XP sm = True, there is unquestionable evidence that 3yP(m,y) holds; namely, there is 
some numeral n such that (P,fh,n) is in s; then (pp is called, and it returns (ppsfh = n. 

This is the basic mechanism by which we implement learning: every state extension is 
linked with an assumption about an instance of EMi which we used and turned out to be 
wrong (this is the only way to come across a counterexample); in next computations, the 
actual state will be bigger, the realizer will not do the same error, and hence will be "wiser" . 

Example 3.8. (il^ formulas) As usual for a Realizability interpretation, we may extract 
from any realizer t III — Vx3y.P(x,y), with P € T, some recursive map ip from the set 
of numerals to the set of numerals, such that P(n,ijj(n)) for all numerals n. Indeed, by 
unfolding the definition of realizer, for all numerals n, all state constants s, iri(tn)[s] lh s 
P(n, TVo(tn)[s]). TTi(tn) has state because t is a realizer. Let us define t(S) = S' if 
and only if ni(tn)\S] = S', and f(S) = S U t(S), as in the proof of the Fixed Point 
Theorem. Set <j>(n) = f k {%) for the first k £ N such that / fc+1 (0) = / fc (0). Then <f>{n) = 
f(4>(n)) = (f>(n) U r((f)(n)), and by ^(n),r(0(n)) disjoint we deduce r((/>(ra)) = 0, that is, 
7Ti(tn)[(f>(n)] = 0. By definition of realizer we have P(n,iro(tn)[(j)(n)]) = True in Tieam- The 
required map tp is then defined by ip(n) = 7To(tn)[(j)(n)] for all numerals n. We may prove 
that the map ip is definable in Tlearn; and even in T, provided we replace the notion of 
convergence used in this paper with the intuitionistic notion of convergence introduced in 
[5], and we use this latter to provide a bound for the first k E N such that f k+1 (S) = f k (S). 
We postpone this topic to another paper. 

Remark 3.9. From the low level computational point of view and in the language of e- 
substitution method, our realizers represent convergent procedures to find out a "solving 
substitution", i.e. a state representing an approximation of Skolem functions (i.e., e-terms) 
which makes true the Skolem axioms instances used in a proof of an existential statement. 
The advantage of our semantics is the possibility of defining such procedures directly from 
high level proofs, by means of Curry-Howard correspondence, hence avoiding the round- 
about route which forces to use a quantifier free deduction system. In the case of a provable 
formula in the language of Peano Arithmetic (that is, one not containing the symbols Xp 
or $p) we do not need at all to modify the language of its proof and to use the Skolem 
axioms \-> ( P- 

Now we explain how to turn each proof P of a formula A € £ciass in HA + EMi into 
a realizers T>* of the same A. By induction on T>, we define a "decoration with realizers" 
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pReai f 2) ; in which each formula B of T> is replaced by a new statement u \- B, for some 
u G 7ciass of state 0. If t h A is the conclusion of T> Rcal , we set P* = t. Then we will prove 
that if T> is closed and without assumptions, then T>* € 7ciass an d T>* I II — A. The decoration 
pReai Q £ <p w j^] 1 rea li zers i s completely standard: we have new realizers only for EMi and 
for atomic formulas. For notation simplicity, if Xi is the label for the set of occurrences of 
some assumption A{ of D, we use Xi also as a name of one free variable in T>* of type \A^\. 
If T is any type of 7s, we denote with d T a dummy term of type T, defined by cF = 0, 

d Bool = FalsS) d S = 0^ d A^B = \_A d B ( with _A any variable Q f type j4 ) ) d AxB = yA ^ ^ 

Definition 3.10. (Term Assignment Rules for HA + EMi). Assume T> is a proof of A G £ciass 
in HA + EMi, with free assumptions A±,..., A n denoted by proof variables x 1 J , . . . , x An and 
free integer variables a\, . . . , a^. By induction onP, we define a decorated proof-tree p Real ; 
in which each formula B is replaced by u \- B for some u £ 7cias S ) and the conclusion ^4 
with some t h A, with FF(t) C {x^ 1 ', . . . , x^,a\, ..., a£j. Eventually we set V* = t. 

(1) x^VA 

if T> consists of a single free assumption A G £ciass labeled x . 
uhA thB uhAAB uhAAB 



(2) 
(3) 
(4) 



(5) 



(6) 



(7) 



{u,t)\~AAB ir u\-A ttiu h B 

uh A-> B t\- A . . uh B 

ut\- B ~Jx^uFA~TB~ 

uh A wh B 

(Tr\ie,u,d B ) \- AV B (False, d A ,u) \- AV B 
uh Ay B Wl hC w 2 \-C 

if Pou then (Xx^ A ^w\)(piu) else (Ax'^'u>2)(p2^) I - C 
where d and d B are dummy closed terms of Tciass of type \A\ and \B\. 

u h MaA u\~ A 

ut h A[t/a] Xa N u \- VaA 

where t is a term of £ciass and a N does not occur free in any free assumption B of the 
subproof of T> of conclusion A. 

(t,u) \-3a».A (Aa N Axl A l t)(7r u)(7Tiu) h C 

where a N is not free in C nor in any free assumption B different from A in the subproof 
of T> of conclusion C. 

u h 4(0) p h Va.Aja) -» A(g(a)) 
Aa N i?uvQ h VaA 
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(8) 



u\ V- A\ u 2 \~ A 2 ■ ■ ■ u n \- A r 



where n > and Ai,A2,...,A n ,A are atomic formulas of Cci ass , and the rule is a Post 
rule for equality or ordering, or a tautological consequence. 



(9) 



h A 



where A is an atomic axiom of HA + EMi (an axiom of equality or of ordering or a 
tautology or an equation of T) 



(10) 



E P h W . 3y P(x, y) V Vy-w^C*, y) 
where P is a predicate of T and £p is defined as Xa ii (Xpd, (&pa, 0), An N Addpan)) 



< n > AddpM h p ( M) ^ B001 x P r ( *- Axiom) 
(12) 0hx P ^ Bool p ( r,(M) (¥>Axiom) 

The term decorating the conclusion of a Post rule is of the form u\ LUJ • • • Ik) u n . In this 
case, we have n different realizers, whose learning capabilities are put together through a 
sort of union. By Lemma l2T2l 2. if u\ W • • • UU u n [s] = 0, then ui[s] = . . . = u n [s] = 0, i.e. 
all Ui "have nothing to learn". In that case, each m must guarantee Ai to be true, and 
therefore the conclusion of the Post rule is true, because true premises Ai, . . . ,A n spell a 
true conclusion A. 

We now prove our main theorem, that every theorem of HA + EMi is realizable. 

Theorem 3.11 (Adequacy Theorem). Suppose that T> is a proof of A in the system HA+EMi 
with free assumptions x x *, . . . , x^™ and free variables a\ : N, . . . , a& : N. Let w = V* . For 
all state constants s and for all numerals m, . . . , Uk, if 

h[s] \\- s Atlm/ai ■■■n k /ak][s},...,t n [s] lh a A n [m/ai ■ ■ ■ n k /a k ] [s] 

then 

w[ti/x[ ll ■■■t n /x^ ni/ai---nk/ak][s]\\- s A[ni/ai---nk/a k ][s} 

Proof. Notation: for any term v and formula B, we denote 

v[ti/x[ ll ---t n /x^ m/ai • • • n k /a k ] [s] 

with v and B[n\/ai ■ ■ ■ n k /a k \\s\ with B. We have \B\ = \B\ for all formulas B. We denote 
with = the provable equality in "/Lcam- We proceed by induction on w. Consider the last 
rule in the derivation V: 

(1) If it is the rule for variables, then w = x\ = x' Al < and A = Ai. So w = ti \\- s Ai = A. 

(2) If it is the Ai rule, then w = (u,t), A = B A C, u h B and t h C. Therefore, 
w = (u,t). By induction hypothesis, ttqW = u lh s B and -K\W = t \\- s C; so, by 
definition, w lh s B A C = A. 

(3) If it is a f\E rule, say left, then w = ttqu and u h A A B. So W = ttqu \\- b A, because 
u \\- s A A B by induction hypothesis. 
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(4) If it is the — > E rule, then w = ut, u b B — >• A and t \- B. So w = ut \\- a A, for 
u \\- s B —?■ A and t lb s B by induction hypothesis. 

(5) If it is the — > I rule, then w = Xx' B 'u, A = B — >■ C and u b C. Thus, HJ = Ax' B 'u. 
Suppose now that i lh s -B; by induction hypothesis on u, wt = u[t/x' B '] lh s C. 

(6) If it is a VI rule, say left, then w = (True, u, d }, j4 = i? V C and u \- B. So, 
u; = (True, u,d ) and hence po^ = True. We indeed verify that p\w = u\\~ s B with 
the help of induction hypothesis. 

(7) If it is a \/E rule, then 

w = if pou then (Ax ^w\)p\u else (Ay' '102 )?W 

and u\~ BV C,wi\~ D,w 2 \~ D,A = D. So, 

w = if pqu then (Ax' 'wJT)(piw) e/se (Ay' 'tyi)(p2^) 

Assume pqu = True. Then by inductive hypothesis p\u \\- s B, and again by induction 
hypothesis, w = wi[piu/x' B '] \\- s D. Symmetrically, if pqu = False, then w lh s D. 

(8) If it is the VE" rule, then w = ut, A = B[t/a] and u b \/aB. So, TZJ = ut. For some 
numeral n we have n = t. By inductive hypothesis « lh s \/aB, therefore ut = un lh s 
5[n/a] = Sp/a] = A. 

(9) If it is the V7 rule, then u; = Xa N u, A = \/aB and u \- B. So, TZJ = Aa N u. Let n be 
a numeral; we have to prove that wn = u[n/a] lb a B[n/a], which is true, indeed, by 
induction hypothesis. 

(10) If it is the BE rule, then w = (Aa N Ax' B li)(7r u)(7riu), t b A and u b Bc^.B. 

Assume n = vrou, for some numeral n. Then 

t[n/o?,niu/x^ n l a ^] lh, A[n/o] = A 

by inductive hypothesis, whose application being justified by the fact, also by induction, 
that u \\~ s 3a N . B and hence 7TiU lb s B[n/a ]. We thus obtain 

uJ = t[7r n/a N 7Tiu/xl B '] lh s A[n/a] = A 

(11) If it is the 3/ rule, then w = (t,u), A = 3aB, u b B[t/a\. So, uJ = (i,tZ); and, indeed, 
7TiuJ = u lb s B[-KQw/a] = B\t/a] since by induction hypothesis u lb s B\t/a\. 

(12) If it is the induction rule, then w = Xa N Ruva, A = \/aB, u b B(0) and v b Va.-B(a) — >• 
B(S(a)). So, uJ = Aa N i?mJa. Now let n be a numeral. A plain induction on n shows 
that wn = Ruvn lh s B[n/a], for u \\- s B(0) and vi lb s S(z) — >■ B(S(i)) for all numerals 
i by induction hypothesis. 

(13) If it is a Post rule, then w = u\ W u 2 UJJ • • • W u n and Uj b .Aj. So, u; = u\ IU) «2 W ■ ■ ■ ^ w n . 
Suppose now that w[s] = 0; then we have to prove that A = True. It suffices to prove 
that A\ = A 2 = ■ ■ ■ = A n = True. By Lemma 12.21 we have u\ = ■ ■ ■ = u n = and by 
induction hypothesis A\ = ■ ■ ■ = A n = True, since U{ \V S A{, for i = 1, . . . , n. 

(14) If it is a x-axiom rule, then w = Addpti . . . t n t and 

A = P(ti, . . . ,tn,t) =^X P t 1 ...t n 

Let t = t\, . . . , t n . For some numeral m we have m = t. Suppose by contradiction 
that W = and P(t,t) = P(t,m) = True and XP s t = False. From XP s ~t = False 
we get (P,t,m') s for all numerals ml. We deduce w = addpstm = {(P,t,m)}, 
contradiction. 

(15) w realizes an EMi axiom: this is Proposition 13.71 



20 F. ASCHIERI AND S. BERARDI 



(16) If it is a ip- axiom rule, then w = and 

A = X P h ...t n ^ P(h, ...,t n , ($ P t! . . . t n )) 

We have w = 0. Let us denote t = t\ . . . t n . Suppose that XP s t = True. Then for some 
numeral m we have {P, t, m) £ s and Ptm = True and (ppst = m. By definition of <pp 
we have 

P(t, (ppst)) = True 

We conclude that A = True. □ 

Corollary 3.12. If A is a closed formula provable in HA + EMi, then there exists t € Tciass 
such that t 1 1 1 — A. 

4. Conclusion and further works 

Many notions of realizability for Classical Logic already exists. A notion similar to our 
one in spirit and motivations is Goodman's notion of Relative realizability [16] . However, 
there is an intrinsic difference between our solution and Goodman's solution. Goodman uses 
forcing to obtain a "static" description of learning. His "possible worlds" are learning states, 
but there is no explicit operation updating a world to a larger word. The dynamic aspect of 
learning (which is represented by a winning strategy in Game Semantics) is therefore lost. 
Using our realizability model, a realizer of an atomic formula, instead of being a trivial map, 
is a map extending worlds, whose fixed points are the worlds in which the atomic formula 
is true. Extending a world represents, in our realizability Semantics, the idea of "learning 
by trial-and-error" that we have in game semantics, while fixed points represent the final 
state of the game. 

A second notion related to our realizability Semantics is Avigad's idea of "update 
procedure" [3]. A state s in our paper corresponds to a finite model of skolem maps in 
Avigad. An "update procedure" is a construction "steering" the future evolution of a finite 
partial model s of skolem maps, to which our individuals belong, in a wanted direction. 
The main difference with our work is that we express this idea formally, by interpreting 
an "update procedure" as a realizer (in the sense of Kreisel) for a Skolem axiom. Another 
important difference is that our realizability relation is defined for all first-order formulas 
with Skolem maps, while the theory of "update procedures" is defined only for quantifier- free 
formulas with Skolem maps. 

Another difference with the other realizability or Kripke models for Classical Logic is 
in the notion of individual and in the equality between individuals. Assume that m is the 
output of a skolem map for 3y.P(n, y), with P decidable, and m = {m[s]\s S S} a family of 
values depending on the finite partial model s. Then our realizer for Skolem axioms "steers" 
the evolution of s towards some universe in which the axiom 3y.P(n,y) =>• P(n,m[s\) is 
true. Modifying the evolution of s may modify the value of m[s]. In our realizability Seman- 
tics we introduce a notion of individuality which is "dynamical" (depending on a state s) 
and "interactive" (the value of the individual depends on what a realizer does). This second 
aspect is new. A realizer may "try" to equate an individual a = {a[s]\s E S} with another 
individual b = {b[s]\s £ S}. Whenever this is possible, the realizer defines a construction 
over the evolution of the universe s producing such an effect, while a random evolution of 
s (without an "interaction" with the realizer) does not guarantee that eventually we have 
a[s] = b[s}. This is why, in our realizability model, even equality among concrete objects is 
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not a "statical" fact, but it is the effect of applying a realizer (which is a construction over 
the evolution of the state or "world" s). In the other models either equality is "static", or, 
even when it is "dynamical" , and it changes with time, it is not "interactive" : the final truth 
value of an equality is not the effect of the application of the realizer, but it is eventually 
the same in all future evolutions of the current world. 

Many aspects of our paper will require some further work. The first author is devel- 
oping in his ph.d. thesis a constructive proof of the Fixed Point Property 12.151 using the 
constructive notion of convergence introduced in [5j. From a foundational viewpoint, this 
result will show that the sub-classical Arithmetic HA+EMi may be subsumed in Intuitionistic 
Arithmetic, in a sense. 

Another challenging idea is to iterate the construction we had for EMi , in order to provide 
a learning model for the entire classical Arithmetic. In this case the leading concepts would 
be the game-theoretical notion of "level of backtracking", introduced in [7] and [9], a notion 
related to the more informal notion of non-monotonic learning. 

Another aspect deserving further work is comparing the programs extracted from clas- 
sical proofs with our method and with other methods, say, with Friedman A-translation. 
Our interpretation, explaining in term of learning how the extracted program work, should 
allow us to modify and improve the extracted program in a way impossible for the more 
formal (but very elegant) ^-translation. 

We remarked that our interpretation is implicitly parametric with respect to the op- 
eration U merging the realizers of two atomic formulas. As explained in [10], by choos- 
ing different variant of this operation we may study different evaluation strategies for the 
extracted programs: sequential and parallel, left-to-right and right-to-left, confluent and 
non-confluent. We would like to study whether by choosing a particular evaluation strategy 
we may extract a more efficient program. 
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